Power-of-d-Choices with Memory: Fluid Limit and Optimality

In multi-server distributed queueing systems, the access of stochastically arriving jobs to resourcesis often regulated by a dispatcher, also known as load balancer. A fundamental problem consists indesigning a load balancing algorithm that minimizes the delays experienced by jobs. During the last twodecades, the power-of-d-choice algorithm, based on the idea of dispatching each job to the least loadedserver out of d servers randomly sampled at the arrival of the job itself, has emerged as a breakthroughin the foundations of this area due to its versatility and appealing asymptotic properties. In this paper,we consider the power-of-d-choice algorithm with the addition of a local memory that keeps track of thelatest observations collected over time on the sampled servers. Then, each job is sent to a server withthe lowest observation. We show that this algorithm is asymptotically optimal in the sense that the loadbalancer can always assign each job to an idle server in the large-server limit. This holds true if and onlyif the system load λ is less than 1− 1d. If this condition is not satisfied, we show that queue lengths arebounded by j + 1, where j ∈ N is given by the solution of a polynomial equation. This is in contrastwith the classic version of the power-of-d-choice algorithm, where queue lengths are unbounded. Ourupper bound on the size of the most loaded server, j∗+1, is tight and increases slowly when λ approachesits critical value from below. For instance, when λ = 0.995 and d = 2 (respectively, d = 3), we find thatno server will contain more than just 5 (3) jobs in equilibrium. Our results quantify and highlight theimportance of using memory as a means to enhance performance in randomized load balancing.